Facet-based conversational search

ABSTRACT

A method is provided. The method comprises receiving, from a user device, a current search query that is in an unstructured format; after receiving the current search query from the user device, retrieving a corresponding current list of facets with a current list of associated values and a context of the current search query, the context comprising a previous search query from the user device and a corresponding previous list of facets with a previous list of associated values; determining a group of actions on one or more of the current list of facets with the current list of associated values and the previous list of facets with the previous list of associated values based on the current search query; creating an updated list of facets with an updated list of associated values based on the group of actions; generating a database query based on the updated list of facets with the updated list of associated values; causing a database search with the database query; transmitting a search result of the database search to the user device, wherein the method is performed by one or more computing devices.

BENEFIT CLAIM

This application claims the benefit under 35 U.S.C. § 119(e) ofprovisional application 62/611,344, filed Dec. 28, 2017, the entirecontents of which is hereby incorporated by reference for all purposesas if fully set forth herein. The applicants hereby rescind anydisclaimer of claim scope in the parent applications or the prosecutionhistory thereof and advise the USPTO that the claims in this applicationmay be broader than any claim in the parent applications.

TECHNICAL FIELD

The present disclosure relates to enabling accurate database searchingbased on natural-language queries, and more specifically toautomatically identifying and predicting facets involved in aconversational search.

BACKGROUND

As more digital information becomes available, it becomes more difficultto find the desired digital information. Many tools have been created tofacilitate a search of digital information. It would be helpful tofurther reduce the amount of user effort required in such a search whileincreasing the accuracy of the search result.

The approaches described in this section are approaches that could bepursued, but not necessarily approaches that have been previouslyconceived or pursued. Therefore, unless otherwise indicated, it shouldnot be assumed that any of the approaches described in this sectionqualify as prior art merely by virtue of their inclusion in thissection.

BRIEF DESCRIPTION OF THE DRAWINGS

In the drawings:

FIG. 1 illustrates an example networked computer system in which variousembodiments may be practiced.

FIG. 2 illustrates an example conversational search server.

FIG. 3 illustrates an example process performed by the conversationalsearch server.

FIG. 4 illustrates an example facet tagging model applied by theconversational search server.

FIG. 5 illustrates an example action tagging process performed by theconversational search server.

FIG. 6 illustrates an example coherence scoring model applied by theconversational search server.

FIG. 7 is a block diagram that illustrates a computer system upon whichan embodiment of the invention may be implemented.

DETAILED DESCRIPTION

In the following description, for the purposes of explanation, numerousspecific details are set forth in order to provide a thoroughunderstanding of the present invention. It will be apparent, however,that the present invention may be practiced without these specificdetails. In other instances, well-known structures and devices are shownin block diagram form in order to avoid unnecessarily obscuring thepresent invention.

GENERAL OVERVIEW

A conversational search server (the “server” hereinafter) and relatedmethods for converting a series of non-structured search queries intoeffective database queries are disclosed. Today's databases can store alarge amount of data, which can be organized and transformed into usefulinformation. In many cases, the more information is included in adescription of an item, the more accurate the description is likely tobe. Having to manage many types of data or information can be achallenge for a user, though. For example, a database management system(“DBMS”) in the recruiting or resource management industry can have coredatabases storing data related to different entities, such as employers,employees, candidates, recruiters, positions, job histories, workrelationships, hiring practices, market trends, or other entitiesrepresented in LinkedIn Economic Graph, for example. Each entity can berepresented by many attributes or facets, such as the name or locationof the employer or the required skills of a job. It can then bedifficult to manage these facets in mentally formulating a search query,one that will find the right candidates for an employer, for instance.Conventional computer tools may facilitate such formulation bypresenting a list of specific facets, for which the user can thenprovide values. However, the sheer number of available facets oftenrenders the display of these facets ineffective, especially when thescreen is small, such as one in a mobile device.

In some embodiments, the server is programmed to enable a user toprovide a search query in a user-friendly form, such as by simplystating what the user thinks he or she is looking for. The server isthen programmed to convert the search query into a form that can beunderstood by a database search engine or a database server within aDBMS. In addition, the server is programmed to engage in a searchconversation (or equivalently “a conversational search” or a “session”)with a user computer by maintaining a context through a series of searchqueries, each submitted after the search result of the previous searchquery is returned.

In some embodiments, a search query can be in one of several forms, suchas a natural language sentence alone or in combination with separatekeywords or specific facet-value pairs. In one example, the presentsearch query can be “Find people working at deep mind in Canada andAustralia”. The server is then programmed to convert the search query byidentifying a first set of facets from the search query, such as byextracting words or phrases from the search query, comparing the wordsor phrases with the data in the core databases or dictionaries, orrunning the words or phrases through a linguistic model. In the example,the first set of facets may contain a first facet of “company” with anassociated value of “deep”, a second facet of “company” with anassociated value of “mind”, a third facet of “company” with anassociated value of “deep mind”, a fourth facet of “location” with anassociated value of “Canada”, and a fifth facet of “location” with anassociated value of “Australia”.

A search query in a search conversation may be concise or partialbecause additional information is contained in previous search queriesand this search query is a follow-up. In some embodiments, the server isprogrammed to further refine the first set of facets into a second setof facets, based on the context of the search query within the searchconversation. Specifically, the server is programmed to determinewhether to add a facet, to drop or modify one of the second set offacets, and so on. In the example, the next search query in the searchconversation may be “Only in Montreal”. The first set of facets in thiscase can consist of the facet of “location” with an associated value of“Montreal”. By considering the previous search query, however, thesecond set of facets can be obtained by supplementing the first set offacets with the facet of “company” with an associated value of “DeepMind”, but not the facet of “location” with an associated value of“Canada”.

In some embodiments, the server is programmed to refine the second setof facets into a third set of facets that are “coherent” orsubstantially likely to lead to a meaningful database search. Suchrefinement can be based on a search history of prior search queriesprovided by a certain group of users, or specifically a search log offacet-value pairs corresponding to prior search queries. Such refinementcan also be based on additional facts or data in the core databases orexternal systems. In the example, the second set of facets may retainthe third facet of “company” with an associated value of “Deep Mind” anda fourth facet of “location” with an associated value of “Canada”because “Deep Mind” is the name of an actual company and it has officesin Canada but not in Australia.

In some embodiments, the server is programmed to identify one or morefacets that can be included in the next search query based on thecontext or the set of prior search queries. In the example, when thesearch history indicates that the search query of “Only in Montreal” isoften followed by “In a particular building” or “Consider Vancouver aswell”, a prediction or suggestion for what to be included in the nextsearch query can be a facet of “building” (without an associated valuebecause prior search queries have covered different values) or a facetof “location” with an associated value of Vancouver.

The server offers several technical benefits. The server reducescomputing requirements. Fewer facets now need to be displayed orotherwise managed through a user interface but can be automaticallyinferred. In addition, fewer and simpler search queries may be receivedfrom a user computer, further reducing network traffic and memory usage.In addition, the server increases the quality of search results. Therefined and inferred facets tend to better characterize the intentbehind the original search query. Furthermore, the server improvescomputer usability. The server allows a user to issue search queriesusing natural language, instead of having to remember various facets ordeal with a crowded user interface presentation. The server furtherallows the user to incrementally change or otherwise adapt their searchqueries, instead of having only one shot at formulating a search queryor having to start from scratch.

Example Computing Environment

FIG. 1 illustrates an example networked computer system in which variousembodiments may be practiced. FIG. 1 is shown in simplified, schematicformat for purposes of illustrating a clear example and otherembodiments may include more, fewer, or different elements connected invarious manners.

In some embodiments, the networked computer system comprises a usersearch manager 124, a conversational search server 102, a databasesearch engine 126, and one or more user computers 122, which arecommunicatively coupled directly or indirectly via one or more networks118. The different components of the networked computer system canreside in the same or different computer network domains.

In some embodiments, the server 102 is programmed to process a searchquery provided by a user and generate a corresponding database query.The server 102 broadly represents one or more computers, virtualcomputing instances, and/or instances of a server-based application thatis programmed or configured with data structures and/or database recordsthat are arranged to host or execute functions of a conversationalsearch server including but not limited to automatically determining andpredicting facets involved in a conversational search. The server 102can comprise a server farm, a cloud computing platform, a parallelcomputer, or any other computing facility with sufficient computingpower in data processing, data storage, and network communication forthe above-described functions.

In some embodiments, a user computer 122 is programmed to transmit asearch query and receive a search result in response to the searchquery. The user computer 122 may comprise a desktop computer, laptopcomputer, tablet computer, smartphone, wearable device, or any othertype of computing device that is capable of proper communication withthe server 102 as well as adequate local data presentation, processing,and storage.

In some embodiments, the user search manager 124 is programmed toreceive a search query provided by a user and return a search result inresponse to the search query. The database search engine 126 isprogrammed to receive a database query, search one or more databasesusing the database query, and return a search result in response to thedatabase query.

The networks 118 may be implemented by any medium or mechanism thatprovides for the exchange of data between the various elements ofFIG. 1. Examples of network 118 include, without limitation, one or moreof a cellular network, communicatively coupled with a data connection tothe computing devices over a cellular antenna, a near-fieldcommunication (NFC) network, a Local Area Network (LAN), a Wide AreaNetwork (WAN), the Internet, a terrestrial or satellite link, etc.

In some embodiments, a user computer 122 is programmed to send a searchquery to the user search manager 124. The user search manager 124 isprogrammed to receive the search query from the user computer 122 andsend the search query to the server 102. The server 102 is programmed toreceive the search query from the user search manager 124, generate adatabase query, and transmit the database query to the database searchengine 126. The database search engine is programmed to receive thedatabase query from the server 102 and return a search result to theserver 102, which can be programmed to return the search result to theuser search manager 124, which can be programmed to return data relatedto the search result to the user computer 122.

In some embodiments, the server 102 is programmed to further generate aprediction or suggestion for what type of information to include in thenext search query and transmit the prediction or suggestion to the usersearch manager 124, which can be programmed to return the prediction orsuggestion to the user computer 122.

In some embodiments, any of the server 102, the user search manager 124,and the database search engine 126 can be integrated into a singlesystem.

Example Components

FIG. 2 illustrates example components of a conversational search server.FIG. 2 is shown in simplified, schematic format for purposes ofillustrating a clear example and other embodiments may include more,fewer, or different elements connected in various manners.

In some embodiments, the server 102 comprises a national languageunderstanding (“NLU”) unit 204, a dialogue manager 208, a databasemanager 210, and an output generator 206.

In some embodiments, the server 102 is programmed to receive a searchquery provided by a user. The search query can be received initially bythe user search manager 124. The user search manger can be a web server,for example, that is programmed to cause display of a graphical userinterface (“GUI”). The interface may have fields that allow a user toprovide a search query, such as a natural-language sentence, keywords,or values for specific facets. The search query can be provided invarious forms, such as speech or text.

When the search query is in the speech form, the server 102 canadditionally be programmed to communicate with an automatic speechrecognition (“ASR”) tool, such as one known to someone with ordinaryskill in the art, that would convert speech to text. The ASR could alsobe incorporated into the user search manager 124.

In some embodiments, the server 102 or specifically the NLU unit 204 isprogrammed to receive a search query in the text form from the usersearch manager 124 or the ASR tool. The NLU unit 204 is programmed tomap the search query to semantic representations that can be consumed bymachines effortlessly. One such semantic representation is a databasequery that can be executed by the database search engine 126 or apreliminary form, such as a set of facet-value pairs together withcertain Boolean operators, leading to a database query.

In some embodiments, the NLU unit 204 is programmed to build alinguistic model that represents a search query. The linguistic modelcan factor in grammars, semantics, or other linguistic features to parsea natural language search query into useful parts. One use of thelinguistic model is to recognize words or sounds that correspond toutterances that indicate mood or intent without actually forming part ofthe search query. For example, the word “hmmm” may indicate hesitation,and the phrase “that's all” or “thank you” may indicate an intent tofinish the current search conversation. Another use of the linguisticmodel is to identify facets that can form a search query. For example,the word “Canada” is associated with a facet of “location”, and thewords “people working at” may be associated with a facet of “employee”.As further discussed below, the facet of “employee” corresponds to anitem to be looked for, while the facet of “location” corresponds to anattribute of an item to be looked for. In other embodiments, the NLUunit 204 is programmed to implement at least a portion of facet tagging,action tagging, and/or coherence scoring, as further discussed below.Performance of each of these tasks can utilize the linguistic modelbuilt by the NLU unit 204 or received from another source as well asother types of data, such as dictionaries, search histories of searchqueries provided by various users, search logs of facet-value pairswhich might corresponding to the search queries, or user profiles.

In some embodiments, the dialogue manager 208 is programmed to maintaincontext details for a search conversation, in which a user computer 122submits a search query, the server 120 returns a search result inresponse to the search query, the user computer 122 submits anothersearch query in response to the search result, and so on. For eachsearch conversation, the dialogue manager 208 can be programmed to trackthe series of search queries, the corresponding sets of facet-valuevalues, or the corresponding user feedback, as further discussed below.In other embodiments, the dialogue manager 208 is programmed toimplement at least a portion of action tagging and facet prediction, asfurther discussed below. In yet other embodiments, certain tasks thatcan be performed by the NLU 204 can be performed by the dialogue manager208, and vice versa.

In some embodiments, the database manager 210 is programmed to managevarious types of data related to the process of converting auser-provided search query to a database query. Some examples include acomprehensive list of facets corresponding to the core databases, one ormore dictionaries, digital representations (includingcomputer-executable instructions) of linguistic models for analyzingsearch queries, different lists of facet-value pairs corresponding toone or more search queries, context information related to a searchconversation including one or more search queries, a list of actionsapplicable to facets, search histories of search queries provided byvarious users, search logs of facet-value pairs corresponding to thesearch queries or input separately, user profiles, training data for thefacet tagging model, action tagging model, coherence scoring model, andface prediction model, and digital representations of these models, asfurther discussed below. In addition to the search queries received viadirection submission to the serer 102, additional actual or plausibleunstructured queries received from another system can be managed by thedatabase manager 210. Similarly, in addition to the search logs derivedfrom the search queries received via direct submission to the server102, additional facet-value pairs that correspond to actual or plausibleunstructured queries received from another system can be managed by thedatabase manager 210.

In some embodiments, the dialogue manager 208 (or the NLU unit 204) isprogrammed to submit a database query to the database search engine 126.The database search engine 126 is programmed to execute the databasequery against the core databases, generate a search result, and transmitthe search result to the dialogue manager 208. In other embodiments, thedatabase search engine 126 can be integrated with the database manger210.

In other embodiments, the dialogue manager 208 or the NLU unit 204 isprogrammed to transmit the search result to the output generator 206.The output generator 206 is configured determine whether and how totransform the search result to a user-friendly form. The search resulttypically includes a set of database records. For example, a record inan employee database can comprise multiple facet-value pairs, such as afacet of “name” with an associated value of “John Smith” and a facet of“gender” with an associated value of “Male”. The output generator 206can be programmed to present the record in a natural language sentence.The output generator 206 can also be programmed to generate a graphicalrepresentation of the search result. In addition, the output generator206 can be programmed to filter the search result to include only theportions of interest to the user who provided the search query. In otherembodiments, the output generator 206 can communicate with atext-to-speech unit, such as one known to someone skilled in the art, toconvert the search result back to speech. The text-to-speech unit isthen programmed to transmit the improved search result in theuser-friendly form back to the user search manager 124. Thetext-to-speech unit can also be integrated into the user search manager124.

In some embodiments, the server 102 is programmed to incorporateadditional user feedback into the process. The output generator 206 canbe programmed to transmit one or more of the different sets offacet-value pairs generated from one search query, to the user searchmanager 124, all at once, once each is available, or according to a userchoice. For example, in response to a natural-language search queryprovided by a user, a first set of facet-value pairs is generatedthrough facet tagging and can be displayed to the user. In response, theuser may indicate certain changes to the first set of facet-value pairs.The user search manger 124 can be programmed to transmit such userfeedback back to the NLU unit 204, which together with the dialoguemanager 208 can be programmed to use the updated set of facet-valuepairs for downstream processing.

Example Processes

FIG. 3 illustrates an example process performed by the conversationalsearch server. FIG. 3 is shown in simplified, schematic format forpurposes of illustrating a clear example and other embodiments mayinclude more, fewer, or different elements connected in various manners.FIG. 3 is intended to disclose an algorithm, plan or outline that can beused to implement one or more computer programs or other softwareelements which when executed cause performing the functionalimprovements and technical advances that are described herein.Furthermore, the flow diagrams herein are described at the same level ofdetail that persons of ordinary skill in the art ordinarily use tocommunicate with one another about algorithms, plans, or specificationsforming a basis of software programs that they plan to code or implementusing their accumulated skill and knowledge.

In some embodiments, the server 102 is programmed to initially createembeddings for words or phrases that are likely to appear in searchqueries or be associated with the words in the search queries, which canfacilitate downstream processing in facet tagging, action tagging, orcoherence scoring. The results of downstream processing can also bebackpropagated to embedding creation to improve the embeddings.Different words or phrases in a given vocabulary appear (or do notappear) in search queries in different manners depending on the specificdomain or industry. Instead of using one-hot vectors where eachdimension corresponds to a word or phrase in the vocabulary, the server102 is programmed to represent the words or phrases with low-dimensionalvectors such that two of these vectors are closer together in the newvector space when they represent similar words or phrases. The creationof embeddings can be performed by building one or more neural networksor utilizing other techniques known to someone skilled in the art. Thecreation of embeddings can be performed in a supervised manner usingsearch histories, search logs, or other actual or plausible unstructuredqueries or facet-value pairs, in an unsupervised manner using hugeamounts of unannotated texts, or a combination of both, such as usingthe result of the unsupervised process to initialize the supervisedprocess. The created embeddings can be saved in an embedding table andlooked up for the words or phrases that appear in search queries in anew search conversation.

In step 302, the server 102 is programmed with data structures and/ordatabase records that are arranged to receive a search query thatincludes natural language, typically from the user search manager 124.For example, the search query can be a natural language phrase orsentence alone or in combination with separate keywords or specificfacet-value pairs. The search query can be part of a searchconversation, where each of a series of search queries follows a searchresult in response to the previous search query.

In step 304, the server 102 is programmed to perform facet tagging. Insome embodiments, the facet tagging can include executing a facettagging model that takes a natural-language or otherwise unstructuredsearch query as input and produces a first list of facet-value pairs asoutput. Specifically, the server 102 is programmed to identify portionsof the search query that correspond to facets or their associatedvalues. In the example given above, when the search query is “Findpeople working at deep mind in Canada and Australia”, the server 102 canbe programmed to recognize that “people” or “people working at”corresponds to “person” as one of the list of predefined facets withoutan associated value in the search query. The server 102 through thefacet tagging model or another component can be further programmed todetermine that the facet of “person” corresponds to the “employee”database (or specifically the “name” field as primary key, for example),or that the facet corresponds to the item to be looked for instead of anattribute of the item. Similarly, the server 102 may be programmed torecognize that “Deep Mind”, “Canada”, and “Australia” correspond to“company”, “location”, and “location” as predefined facets with theirassociated values. For “Deep Mind”, for example, the server 102 throughthe facet tagging model or another component can be further programmedto determine that the facet corresponds to the “employer” field of the“employee” database or that the facet corresponds to an attribute of theitem to be looked for.

In some embodiments, the server 102 is programmed to further determinedomain-specific relationships. In the recruiting or resource managementindustry, for example, some of the search queries may concern acandidate's employment history. The server 102 can thus be programmed torecognize that certain words or phrases in the search query, such as“worked before”, “used to work”, or “previous job”, can correspond tospecific facets related to employment history, such as the “priorcompany” field or the combination of the “company” and “time ofemployment” fields of the employee database.

In some embodiments, the facet tagging model can be trained with machinelearning techniques using search histories of prior search queries,synthetic natural-language queries derived from search logs offacet-value pairs or input separately, or other actual or plausibleunstructured queries, which correspond to the model input. The facettagging model can be further trained with the facet-value pairs that canbe extracted from those unstructured queries, which correspond to themodel output. The training or execution of the facet tagging model canutilize the initially computed embeddings to represent the unstructuredqueries (e.g., embeddings for the words or phrases in an unstructuredquery) or facet-value pairs (e.g., embeddings for the facets). As moreunstructured queries and corresponding facet-value pairs becomeavailable, the facet tagging model can be retrained. Such retraining canoccur periodically, upon specific request, or when the number of newunstructured queries available for training reaches a certain threshold.

FIG. 4 illustrates an example facet tagging model applied by theconversational search server. In some embodiments, the server 102 isprogrammed to generate a list of spans from a search query in anunstructured form via the span generation component 402. Each span is acontiguous portion of the search query that corresponds to a facet.Specifically, the span generation component 402 can include identifyingeach contiguous portion of the search query and determining thecorresponding role of the contiguous portion, such as a facet. In theexample given above, of the search query “Find people working at deepmind in Canada and Australia”, one contiguous portion is “and”, anothercontiguous portion is “deep”, and yet another contiguous portion is“deep mind”, with the last two being spans. Other roles include a facetrelationship indicator, an action indicator, or a signal for starting orending a search conversation, as further discussed below. The server 102is programmed to next generate a list of facet-value pairs from the listof spans via the facet tagging component 404. In the example givenabove, the span of “deep” and the span of “deep mind” each correspond toa facet of “company”, for example.

In some embodiments, the span generation component 402 and/or the facettagging component 404 can be implemented using one or more of rule-basedoperators, logistic regression classifiers, recurrent, feedforward, orother types of neural networks that represent or characterize differentparts of an actual or plausible unstructured query, and othercomputational techniques. For example, a convolutional neural network(“CNN”) that performs part-of-speech-tagging may be adapted to recognizefacets directly from an unstructured query. Specifically, to tag a spanwithin a search query, the CNN may accept as input a concatenation of anembedding of the span and the embedding of the search query, whichcollectively encode characteristics of how the span fits in the searchquery, and produce as output a distribution of how likely it is that thespan corresponds to each of the possible facets, with the facetassociated with the highest probability assigned to the span.

In step 306, the server 102 is programmed to perform action tagging. Insome embodiments, the action tagging can include executing an actiontagging model that takes the search query, the first list of facet-valuepairs (the value could be null for certain facets), and the context ofthe search query within the search conversation as input and produces asecond list of facet-value pairs that is meaningful with respect to theprevious search queries in the search conversation as output. Thecontext can include a series of search queries, each except for thefirst one made after a search result in response to the previous searchquery is returned, the lists of facet-value pairs for each of the searchqueries, or the search result in response to each of the search queries.The search result can be used to determine whether the facetscorresponding to the last search query were coherent, for instance, asfurther discussed below. Specifically, the server 102 is programmed toconvert the first list of facet-value pairs to the second list offacet-value pairs to ensure an appropriate level of information content.Such conversion to the second list of facet-value pairs generallyincludes identifying an action, such as addition, removal, replacement,or update, for a facet-value pair in the first list or in the list offacet-value pairs corresponding to the last search query in the searchconversation. Specifically, the conversion to the second list offacet-value pairs could start with the facet-value pairs in the firstlist and consider removing facet-value pairs or adding certain facetvalue pairs corresponding to the last search query, or (the other wayaround) start with the facet value pairs corresponding to the lastsearch query and consider removing some facet-value pairs or addingfacet value pairs from the first list.

In some embodiments, the server 102 is programmed to identify indicatorsfor an action from the search query. Such indicator identification canborrow some operations from span generation discussed above or reusesome results of the span generation. Certain signals in a search query,such as “only”, “instead of”, “both”, “but”, or other conjunction wordsor phrases, often indicate specific actions. In the example given above,the word “only” can be identified from the search query “Only inMontreal” as leading to no action for a facet of “location”corresponding to the previous search query (and addition for the otherfacet-value pairs corresponding to the previous search query) as thatfacet is being overridden by the facet with an associated value ofMontreal. These signals can also indicate relationships between facets,as further discussed below. In the example above, when the currentsearch query is “In Spain as well”, the signal “as well” indicates thatthe facet of location with an associated value of “Spain” have aconjunctive relationship with the two existing facets of location withassociated values of Canada and Australia, respectively. Other signalsin a search query can be verbs that directly indicate specific actions.In the example given above, a search query of “Eliminate Australia fromconsideration” indicates that the action should be a removal for thefacet of “location” with an associated value of “Australia” in the firstlist.

In other embodiments, the server 102 is programmed to identifyindicators for an action by considering previous search queries in thesearch conversation and related data. Specifically, known relationshipsbetween different facets respectively corresponding to the currentsearch query and the previous search query in the search conversationcan be taken into consideration, such as one facet (e.g. “country”)being broader than another (e.g. “city”). In certain embodiments, whenthere is no relationship (and the facet-value pair in the first list offacet-value pairs does not correspond to the items to be looked for),the server 102 can be programmed to assign an action of addition to eachof the facet-value pairs corresponding to the previous search query.When there is a relationship, the server 102 can be programmed todetermine which actions to assign to the related facets depending on thenature of the relationship. In the example given above, when the searchquery following “Find people working at deep mind in Canada andAustralia” is “Only in Montreal”, there would be no action for the facetof “location” with an associated value of Canada, as the facet of“location” with an associated value of Montreal is being added. If thefacet with an associated value of Canada was “country” instead, theaction for the facet could be an addition, given that the facet of“city” with an associated value of Montreal is being added. In addition,some other facets corresponding to the previous search query may need tobe incorporated to provide some context for the present search query.There would similarly be no action for facet of “location” with anassociated value of Australia, but the action for the facet of “company”with an associated value of “Deep Mind” would be an addition.

In some embodiments, the server 102 is programmed to recognize that thepresent search query is unrelated to the previous search query receivedfrom the same sender, thus marking an end to an existing searchconversation and a beginning to a new search conversation. The server102 can be programmed to determine the relationship between the firstlist of facet-value pairs and the facet-value pairs corresponding to oneor more previous search queries, especially the relationship betweenfacets that correspond to items to be searched for. For example, whenthe previous search query is “Find people working at deep mind in Canadaand Australia” and the present search query is “Find companies that arelocated in Wisconsin”, the facet of “people” that corresponds to theitems to be searched for in the previous search query is unrelated tothe facet of “company” that corresponds to the item to be searched forin the present search query. Therefore, the present search query isdeemed as starting a new search conversation. For further example, whenthe previous search query is “Find people working at deep mind in Canadaand Australia” but the present search query is “Find people working atapple in the United States”, the facet of “people” that corresponds toitems to be searched for is shared. In the presence of certain signalsin the present search query indicating a relationship with the previoussearch query, such as “now”, “instead”, or “start over”, the presentsearch query can also be deemed to start a new search conversation.

In some embodiment, the server 102 is programmed to next generate a listof actions for facet-value pairs from a list of facet-value pairscorresponding to the present search query, a list of facet-value pairscorresponding to the previous search query. The application of theactions leads to an updated list of facet-value pairs. In the examplegiven above, based on a review of the list of facet-value pairscorresponding to the previous search query “Find people working at deepmind in Canada and Australia” and the first list of facet-value pairscorresponding to the present search query “Only in Montreal”, it may bedetermined that the facet of “country” with an associated value ofCanada is related to the facet of “city” with an associated value ofMontreal. The action of addition may then be assigned to the facet of“country” with an associated value of Canada as the facet of “country”is broader than the facet of “city”. If both facets were “location”instead, no action may be assigned to the facet with an associated valueof Canada. The signal of “only” in the search query is used to furtherindicate the relationship between the facet with an associated value ofCanada and the facet with an associated value of Montreal. If the signalhad been “except for” instead of “only”, for instance, the facet with anassociated value of Montreal would need to be associated with a negativeoperator (e.g., “not”), and the facet with an associated value of Canadawould need to be added, or alternatively the facet with an associatedvalue of Montreal would need to be removed and a facet with anassociated value of all cities in Canada except for Montreal would needto be added. When the present search query is “Find companies that arelocated in Wisconsin” instead, the last search conversation would beterminated, a new search conversation would be started withreinitialized context information.

In some embodiments, the action tagging model can be trained usingsequences of search queries in search histories, sequences of syntheticnatural-language queries derived from sequences of lists of facet-valuepairs corresponding to the sequences of prior search queries or inputseparately, or sequences of other actual or plausible unstructuredqueries, as well as the corresponding lists of facet-value pairs in thesequences, which correspond to the model input. The action tagging modelcan be further trained with certain lists of facet-value pairs withassociated action labels that can be derived from those sequences ofunstructured queries and the corresponding lists of facet-value pairs,which correspond to the model output. The training or execution of theaction tagging model can utilize the initially computed embeddings torepresent the unstructured queries or facet-value pairs. As moresequences of unstructured queries and corresponding facet-value pairsand action labels become available, the action tagging model can beretrained. Such retraining can occur periodically, upon specificrequest, or when the number of new search sequences of unstructuredqueries and corresponding facet-value pairs and action labels availablereaches a certain threshold. Action tagging can be implemented using oneor more of rule-based operators, recurrent, feed forward, or other typesof neural networks that represent or characterize differentrelationships between actual or plausible facet-value pairs, and othercomputational techniques. For example, a feed forward neural network mayaccept as input a concatenation of an embedding of a facet-value pair tobe classified, which can belong to the first list or correspond to thelast search query, and the embedding of the search query, whichcollectively encode characteristics of how the facet-value pair isrelated to the search query, and produce as output a distribution of howlikely it is that the facet-value pair is subject to each of thepossible actions, with the action associated with the highestprobability assigned to the facet-value pair.

FIG. 5 illustrates an example action tagging process performed by theconversational search server. In some embodiments, in step 502, theserver 102 is programmed to receive a current search query provided by auser device that is in an unstructured format, a corresponding currentlist of facets with a current list of associated values, and a contextof the current search query. The context may comprise at least oneprevious search query from the user device and a corresponding previouslist of facets with a previous list of associated values. When thecurrent search query extends a current search conversation, the contextmay comprise a series of previous search queries from the user deviceand related data in the current search conversation.

In step 504, the server 102 is programmed to determine a group ofactions on one or more of the current list of facets with the currentlist of associated values and the previous list of facets and theprevious list of associated values based on the current search query. Insuch determination, the server 102 can be configured to identify a listof action signals from the current search query, such as conjunctions orverbs, which may describe Boolean relations among facets with theirassociated values or specify actions on facets with their associatedvalues. The server 102 can also be configured to identify a set ofknown, scope-based relations specifically among the current list offacets and the previous list of facets, such as one facet of “country”being broader than another facet of “city”.

In step 506, the server 102 is programmed then to create an updated listof facets with an updated list of associated values based on the groupof actions. For example, the group of actions may be to start with thecurrent list of facets with the current list of associated values, addthe previous list of facets with the previous list of associated valuessubject to appropriate Boolean relations, and for each of the previouslist of facets that is related to one of the current list of facetsscope-wise, remove, replace, or update that facet with its associatedvalue. Applying the group of actions then leads to the updated list offacets with the updated list of associated values, from which a databasequery can ultimately be created and executed to generate search resultsin response to the current search query.

In step 308, the server 102 is programmed to perform coherence scoring.In some embodiments, the coherence scoring can include executing acoherence scoring model that takes the second (or the first) list offacet-value pairs (the value could be null for certain facets) as inputand produces a third list of facet-value pairs having a high coherencescore as output. Specifically, the server 102 is programmed to convertthe second list of facet-value pairs to the third list of facet-valuepairs to maximize a coherence score—the likelihood that the facet-valuepairs lead to a meaningful search. Such meaningfulness can be measuredagainst search histories, search logs, the core databases, and othersources of facts. Generally, a search that returns an empty searchresult is not considered meaningful, while a search based on facet-valuepairs that have appeared frequently in the search logs may be consideredmeaningful. In the example given above, the server 102 can be programmedto recognize that inclusion of the facet of “location” with anassociated value of “Australia” will lead to an empty search resultbecause the company Deep Mind does not have any office in Australia. Theserver 102 might also be programmed to recognize that while othercompanies exist that have offices in both Canada and Australia, thefacet of “company” is associated with a larger weight than the facet of“location” or that the search for people working in Deep Mind's Canadaoffice is associated with a higher interest among users.

In some embodiments, the coherence scoring model can be trained usingsearch logs as well as search log derivatives obtained from randomlyperturbing portions of the search logs or input separately, or otheractual or plausible lists of facet-value values, which correspond to themodel input. The coherence scoring model can be further trained withassociated coherence scores, which correspond to the model output. Thetraining or execution of the coherence scoring model can utilize theinitially computed embeddings to represent the facet-value pairs. Asmore lists of facet-value pairs and corresponding coherence scoresbecome available, the coherence scoring model can be retrained. Suchretraining can occur periodically, upon specific request, or when thenumber of new search queries available reaches a certain threshold.

Specifically, in training the coherence scoring model, each of a groupof lists of facet-value pairs from the search logs or otherwisecorresponding to actual or plausible unstructured queries can beassigned a high coherence score. Each of the lists of facet-value pairscan then be converted to one or more lists of facet-value pairs that arethen each assigned a low coherence score for deviating from an actual orplausible unstructured query. The conversion can involve randomlyreplacing or updating one of the facet-value pairs, or adding a newfacet-value pair, for example. The nature of the conversion candetermine the value of the coherence score. A random change typicallyleads to a new list of facet-value pairs that does not correspond to anyactual unstructured query, and so the new list can be assigned a lowercoherence value. When the new list of facet-value pairs also does notcorrespond to a plausible unstructured query based on other sources ofdata, such as the core databases or certain dictionaries, then the newlist can be assigned an even lower coherence score. For example, given alist of facet-value pairs, including a facet of “company” with anassociated value of Deep Mind and a facet of “location” with anassociated value of Montreal, some perturbations include changing DeepMind to Deep Soul, replacing the facet of “location” with an associatedvalue of Montreal by a facet of “function” and an associated value ofsale, or adding a facet of “color” with an associated value of red. Thefacet of “color” with an associated value of red may be consideredespecially incoherent with the original list of facet value pairs.

In other embodiments, the server 102 is programmed to perform coherencescoring before action tagging, or perform coherence scoring repeatedly,such as after facet tagging and action tagging.

FIG. 6 illustrates an example coherence scoring model applied by theconversational search server. In some embodiments, the server 102 isprogrammed to compute a coherence score for each subset of a list offacet-value pairs via the facet-value set scoring component 602. As thecomputation for substantially all possible subsets of the list offacet-value pairs generally satisfies the overlapping subprogram orsubstructure property, it can be performed via dynamic programming.Alternatively, the server 102 can be programmed to compute a coherencescore for select subsets of the list of facet-value pairs using beamsearch or other heuristic search algorithms. The server 102 is furtherprogrammed to select one or more subsets of facet-value pairs fordownstream processing based on the coherence scores via the facet-valueselection component 604. The server 102 can be programmed to select afixed number or percentage of highest-scoring subsets of facet-valuepairs, select those subsets of facet-value pairs with their coherencescores above a fixed threshold, or use other related techniques.

In some embodiments, the facet-value set scoring component 602 and/orthe facet-value selection component 604 can be implemented using one ormore of rule-based operators, logistic regression classifiers,recurrent, feedforward, or other types of neural networks that representor characterize a list of facet-value pairs corresponding to an actualor plausible unstructured query, and other computational techniques.

In step 310, the server 102 is programmed to formulate a database query.For example, the formulation of a SQL query can involve creating aselect clause for the facet without an associated value and a whereinclause for each facet having an associated value. In some embodiments,the server 102 can also be programmed to keep track of the relationshipsamong the different facets during facet tagging, action tagging,coherence scoring, or other processing, which would then be translatedinto relationship operators in the database query. More specifically,typical words or phrases that indicate Boolean relationships in thesearch query can be taken as indicators for relationships betweenfacets. Some examples of such words or phrases are “and”, “or”, “aswell”, “except”, “but”, “not”, “additional”, and “together with”. Therelationships between search queries can also be taken as suchindicators. For example, typically, the facet-value pairs added based onprevious search queries are to be considered conjunctively with thefacet-value pairs in the present search query. The server 102 isprogrammed to further submit the database query to the database searchengine 126 for execution against the core databases. In step 312, theserver 102 is programmed to receive the search result of the databasequery. In other embodiments, the server 102 is programmed to transmitthe search result to the user search manager 124, which can beprogrammed to cause display of the search result.

In addition, the server 102 can be programmed to perform facetprediction. In some embodiments, the facet prediction can includeexecuting a facet prediction model that takes as input one or moresearch queries in the search conversation and produces as output one ormore facets with associated values (which can be null) to include in thenext search query in the search conversation. The generation orselection of one or more facets with associated values can be based onsearch logs of facet-value pairs corresponding to prior search queries,relationships among entities captured in the core databases, or otherdata sources.

In the example given above, given the search queries received so far inthe search conversation are “Find people working at deep mind in Canadaand Australia” and “Only in Montreal”, the server 102 can be programmedto suggest referring to a facet of “location” with a value of“Vancouver” in the next search query because a search query that refersto a facet of “location” with an associated value of “Vancouver” isoften received after a search query similar to “Only in Montreal” due tothe small number of people in the Montreal office, the fact thatemployees often move between the two offices, etc. The server 102 canalso be programmed to suggest referring to a facet of “building” in thenext search query (without a suggestion for an associated value) becausethere are many buildings in the Montreal office and generally a facet of“building” is referenced early on a search conversation in searchconversations.

In some embodiments, the server 102 is programmed to utilize the resultof coherence scoring in facet prediction. For example, instead ofremoving a facet because it appears incoherent with the rest of thefacets, the server 102 can be programmed to keep that facet in buildinga database query for the present search query but suggest removing thefacet in the next search query. Similarly, the server 102 can take intoconsideration the quality of the search result in facet prediction. Whenthe quality of the search result is low because the database match scoreis low or the number of hits is too large or too small, the server 102can be programmed to assist the user with improving the quality of thesearch result by eliminating incoherent facet-value pairs, or adding orremoving facet-value pairs to broaden or narrow the scope of the search.In other embodiments, the server 102 is programmed to provide anexplanation for the suggestion of a facet. Therefore, the user computer122 can receive not only a suggestion for a facet-value pair to beincluded in the next search query but also the reason for suchinclusion.

In some embodiments, the server 102 is programmed to incorporate userinput into facet prediction. Specifically, the server 102 is programmedto initially produce several candidate facets for inclusion in the nextsearch query and cause display of the several candidate facets for userselection. Alternatively, the server 102 can be programmed to converteach of those candidate facets into a corresponding natural-languagequery and cause display of those natural-language queries. Furthermore,the server 102 is programmed to then take the user's selection,construct a full list of facet-value pairs, convert it into a databasequery, and carry the search conversation forward. Such facet predictionfurther reduces the need to display any large number of facets or savesboth computer resources and human efforts.

In some embodiments, the facet prediction model can be trained usingsearch logs of lists of facet-value pairs corresponding to searchqueries in prior search conversations or sequences of other actual ofplausible unstructured queries, which correspond to the model input. Thefacet prediction model can be further trained with additionalfacet-value pairs in subsequent search queries in those prior searchconversations or subsequent unstructured queries in those sequences ofunstructured queries, which correspond to the model output. The trainingor execution of the facet prediction model can utilize the initiallycomputed embeddings to represent the facet-value pairs. As more searchlogs or lists of facet-value pairs become available, the facetprediction model can be retrained. Such retraining can occurperiodically, upon specific request, or when the number of new lists offacet-value pairs available reaches a certain threshold.

In some embodiments, the facet prediction model can be implemented usingone or more of rule-based operations, recurrent or other types of neuralnetworks that represent or characterize a series of lists of facet-valuepairs corresponding to a series of search queries in a searchconversation or a sequence of actual or plausible unstructured queries,and other computational techniques for text generation. For example, arule-based approach can be used to generate one or more new facet-valuepairs given the facet-value pairs corresponding to the last searchquery. More broadly, a recurrent neural network (RNN) with longshort-term memory (LSTM) units can be used to generate one or more newfacet-value pairs given a list of facet-value pairs corresponding toprevious search queries received so far in the search conversation.

Hardware Overview

According to one embodiment, the techniques described herein areimplemented by one or more special-purpose computing devices. Thespecial-purpose computing devices may be hard-wired to perform thetechniques, or may include digital electronic devices such as one ormore application-specific integrated circuits (ASICs) or fieldprogrammable gate arrays (FPGAs) that are persistently programmed toperform the techniques, or may include one or more general purposehardware processors programmed to perform the techniques pursuant toprogram instructions in firmware, memory, other storage, or acombination. Such special-purpose computing devices may also combinecustom hard-wired logic, ASICs, or FPGAs with custom programming toaccomplish the techniques. The special-purpose computing devices may bedesktop computer systems, portable computer systems, handheld devices,networking devices or any other device that incorporates hard-wiredand/or program logic to implement the techniques.

For example, FIG. 7 is a block diagram that illustrates a computersystem 700 upon which an embodiment of the invention may be implemented.Computer system 700 includes a bus 702 or other communication mechanismfor communicating information, and a hardware processor 704 coupled withbus 702 for processing information. Hardware processor 704 may be, forexample, a general purpose microprocessor.

Computer system 700 also includes a main memory 706, such as a randomaccess memory (RAM) or other dynamic storage device, coupled to bus 702for storing information and instructions to be executed by processor704. Main memory 706 also may be used for storing temporary variables orother intermediate information during execution of instructions to beexecuted by processor 704. Such instructions, when stored innon-transitory storage media accessible to processor 704, rendercomputer system 700 into a special-purpose machine that is customized toperform the operations specified in the instructions.

Computer system 700 further includes a read only memory (ROM) 708 orother static storage device coupled to bus 702 for storing staticinformation and instructions for processor 704. A storage device 710,such as a magnetic disk, optical disk, or solid-state drive is providedand coupled to bus 702 for storing information and instructions.

Computer system 700 may be coupled via bus 702 to a display 712, such asa cathode ray tube (CRT), for displaying information to a computer user.An input device 714, including alphanumeric and other keys, is coupledto bus 702 for communicating information and command selections toprocessor 704. Another type of user input device is cursor control 716,such as a mouse, a trackball, or cursor direction keys for communicatingdirection information and command selections to processor 704 and forcontrolling cursor movement on display 712. This input device typicallyhas two degrees of freedom in two axes, a first axis (e.g., x) and asecond axis (e.g., y), that allows the device to specify positions in aplane.

Computer system 700 may implement the techniques described herein usingcustomized hard-wired logic, one or more ASICs or FPGAs, firmware and/orprogram logic which in combination with the computer system causes orprograms computer system 700 to be a special-purpose machine. Accordingto one embodiment, the techniques herein are performed by computersystem 700 in response to processor 704 executing one or more sequencesof one or more instructions contained in main memory 706. Suchinstructions may be read into main memory 706 from another storagemedium, such as storage device 710. Execution of the sequences ofinstructions contained in main memory 706 causes processor 704 toperform the process steps described herein. In alternative embodiments,hard-wired circuitry may be used in place of or in combination withsoftware instructions.

The term “storage media” as used herein refers to any non-transitorymedia that store data and/or instructions that cause a machine tooperate in a specific fashion. Such storage media may comprisenon-volatile media and/or volatile media. Non-volatile media includes,for example, optical disks, magnetic disks, or solid-state drives, suchas storage device 710. Volatile media includes dynamic memory, such asmain memory 706. Common forms of storage media include, for example, afloppy disk, a flexible disk, hard disk, solid-state drive, magnetictape, or any other magnetic data storage medium, a CD-ROM, any otheroptical data storage medium, any physical medium with patterns of holes,a RAM, a PROM, and EPROM, a FLASH-EPROM, NVRAM, any other memory chip orcartridge.

Storage media is distinct from but may be used in conjunction withtransmission media. Transmission media participates in transferringinformation between storage media. For example, transmission mediaincludes coaxial cables, copper wire and fiber optics, including thewires that comprise bus 702. Transmission media can also take the formof acoustic or light waves, such as those generated during radio-waveand infra-red data communications.

Various forms of media may be involved in carrying one or more sequencesof one or more instructions to processor 704 for execution. For example,the instructions may initially be carried on a magnetic disk orsolid-state drive of a remote computer. The remote computer can load theinstructions into its dynamic memory and send the instructions over atelephone line using a modem. A modem local to computer system 700 canreceive the data on the telephone line and use an infra-red transmitterto convert the data to an infra-red signal. An infra-red detector canreceive the data carried in the infra-red signal and appropriatecircuitry can place the data on bus 702. Bus 702 carries the data tomain memory 706, from which processor 704 retrieves and executes theinstructions. The instructions received by main memory 706 mayoptionally be stored on storage device 710 either before or afterexecution by processor 704.

Computer system 700 also includes a communication interface 718 coupledto bus 702. Communication interface 718 provides a two-way datacommunication coupling to a network link 720 that is connected to alocal network 722. For example, communication interface 718 may be anintegrated services digital network (ISDN) card, cable modem, satellitemodem, or a modem to provide a data communication connection to acorresponding type of telephone line. As another example, communicationinterface 718 may be a local area network (LAN) card to provide a datacommunication connection to a compatible LAN. Wireless links may also beimplemented. In any such implementation, communication interface 718sends and receives electrical, electromagnetic or optical signals thatcarry digital data streams representing various types of information.

Network link 720 typically provides data communication through one ormore networks to other data devices. For example, network link 720 mayprovide a connection through local network 722 to a host computer 724 orto data equipment operated by an Internet Service Provider (ISP) 726.ISP 726 in turn provides data communication services through the worldwide packet data communication network now commonly referred to as the“Internet” 728. Local network 722 and Internet 728 both use electrical,electromagnetic or optical signals that carry digital data streams. Thesignals through the various networks and the signals on network link 720and through communication interface 718, which carry the digital data toand from computer system 700, are example forms of transmission media.

Computer system 700 can send messages and receive data, includingprogram code, through the network(s), network link 720 and communicationinterface 718. In the Internet example, a server 730 might transmit arequested code for an application program through Internet 728, ISP 726,local network 722 and communication interface 718.

The received code may be executed by processor 704 as it is received,and/or stored in storage device 710, or other non-volatile storage forlater execution.

In the foregoing specification, embodiments of the invention have beendescribed with reference to numerous specific details that may vary fromimplementation to implementation. The specification and drawings are,accordingly, to be regarded in an illustrative rather than a restrictivesense. The sole and exclusive indicator of the scope of the invention,and what is intended by the applicants to be the scope of the invention,is the literal and equivalent scope of the set of claims that issue fromthis application, in the specific form in which such claims issue,including any subsequent correction.

What is claimed is:
 1. A method comprising: receiving, from a user device, a current search query that is in an unstructured format; determining a current list of facet-value pairs and a context of the current search query; the context comprising a previous search query received from the user device and a previous list of facet-value pairs; determining at least one action to perform on at least one of a facet-value pair of the current list of facet-value pairs and a facet-value pair of the previous list of facet-value pairs by (i) creating neural network inputs including an embedding of the current search query and an embedding of at least one of the facet-value pair of the current list of facet-value pairs and the facet-value pair of the previous list of facet-value pairs, (ii) applying a trained neural network to the neural network inputs, (iii) by the trained neural network, generating and outputting a distribution that comprises, for each of a plurality of different actions, a likelihood of a particular facet-value pair being subject to the action, wherein the plurality of different actions comprises adding a facet-value pair, removing a facet-value pair, updating a value associated with a facet, and replacing a facet-value pair with a new facet-value pair, and (iv) based on the distribution, assigning an action of the plurality of different actions to the particular facet-value pair; performing the action on the particular facet-value pair; creating an updated list of facet-value pairs; generating a database query based on the updated list of facet-value pairs; causing a database search with the database query; and transmitting a search result of the database search to the user device.
 2. The method of claim 1, further comprising: identifying, from the current search query, a list of action signals indicating at least one action on one of the current list of facet-value pairs and the previous list of facet-value pairs or on a search conversation with the user device comprising the current search query and the previous search query; and identifying a list of relations among the current list of facet-value pairs and the previous list of facet-value pairs.
 3. The method of claim 1, the at least one action including adding a facet and an associated value, removing a specific facet and an associated value, updating a value associated with a particular facet, or replacing a certain facet and an associated value with a new facet and an associated value.
 4. The method of claim 1, further comprising: combining the previous list of facet-value pairs and the current list of facet-value pairs; and removing, updating, or replacing at least one facet-value pair of the previous list of facet-value pairs that is related to at least one facet-value pair of the current list of facet-value pairs.
 5. The method of claim 1, further comprising: deciding whether the current search query extends a current search conversation comprising the previous search query or starts a new search conversation with the user device, the determining, the creating, and the generating being performed in response to a decision that the current search query extends the current search conversation; generating a database query based on the current list of facets with the current list of associated values in response to a decision that the current search query starts a new conversation.
 6. The method of claim 1, further comprising: receiving a plurality of search logs from search conversations with a plurality of other user devices, each search log comprising a series of lists of facet-value pairs; predicting a next list of facet-value pairs from the current list of facet-value pairs, the previous list of facet-value pairs, and the plurality of search logs.
 7. The method of claim 6, further comprising: generating a next search query from the next list of facet-value pairs; transmitting the next search query to the user device.
 8. The method of claim 6, further comprising: transmitting the next list of facet-value pairs to the user device; and receiving changes to the next list of facet-value pairs from the user device.
 9. The method of claim 1, further comprising: concatenating the embedding of the current search query and the embedding of at least one of the facet-value pair of the current list of facet-value pairs and the facet-value pair of the previous list of facet-value pairs.
 10. The method of claim 1, wherein the trained neural network comprises a feed forward neural network and the method further comprises: applying the feed forward neural network to the neural network inputs; and by the feed forward neural network, outputting the distribution.
 11. The method of claim 1, further comprising: extracting sequences of unstructured queries from search histories; deriving action labels from the sequences of unstructured queries; and training the trained neural network on lists of facet-value pairs associated with the action labels.
 12. One or more non-transitory storage media storing instructions which, when executed by one or more computing devices, cause the one or more computing devices to perform operations comprising: receiving, from a user device, a current search query that is in an unstructured format; determining a current list of facet-value pairs and a context of the current search query; the context comprising a previous search query received from the user device and a previous list of facet-value pairs; determining at least one action to perform on at least one of a facet-value pair of the current list of facet-value pairs and a facet-value pair of the previous list of facet-value pairs by (i) creating neural network inputs including an embedding of the current search query and an embedding of at least one of the facet-value pair of the current list of facet-value pairs and the facet-value pair of the previous list of facet-value pairs, (ii) applying a trained neural network to the neural network inputs, (iii) by the trained neural network, generating and outputting a distribution that comprises, for each of a plurality of different actions, a likelihood of a particular facet-value pair being subject to the action, wherein the plurality of different actions comprises adding a facet-value pair, removing a facet-value pair, updating a value associated with a facet, and replacing a facet-value pair with a new facet-value pair, and (iv) based on the distribution, assigning an action of the plurality of different actions to the particular facet-value pair; performing the action on the particular facet-value pair; creating an updated list of facet-value pairs; generating a database query based on the updated list of facet-value pairs; causing a database search with the database query; and transmitting a search result of the database search to the user device.
 13. The one or more non-transitory storage media of claim 12, wherein the instructions cause the one or more computing devices to perform operations further comprising: identifying, from the current search query, a list of action signals indicating at least one action on one of the current list of facet-value pairs and the previous list of facet-value pairs or on a search conversation with the user device comprising the current search query and the previous search query; and identifying a list of relations among the current list of facet-value pairs and the previous list of facet-value pairs.
 14. The one or more non-transitory storage media of claim 13, wherein the list of action signals includes conjunction words or phrases or verbs.
 15. The one or more non-transitory storage media of claim 13, wherein the list of action signals indicates a relationship between multiple of the current list of facet-value pairs and the previous list of facet-value pairs.
 16. The one or more non-transitory storage media of claim 12, wherein the instructions cause the one or more computing devices to perform operations further comprising: identifying a list of relations among the current list of facet-value pairs and the previous list of facet-value pairs from an existing database.
 17. The one or more non-transitory storage media of claim 12, wherein the at least one action includes at least one of adding a facet-value pair, removing a facet-value pair, updating a value associated with a facet, and replacing a facet-value pair with a new facet-value pair.
 18. The one or more non-transitory storage media of claim 12, wherein the instructions cause the one or more computing devices to perform operations further comprising: combining the previous list of facet-value pairs and the current list of facet-value pairs; and removing, updating, or replacing at least one facet-value pair of the previous list of facet-value pairs that is related to at least one facet-value pair of the current list of facet-value pairs.
 19. The one or more non-transitory storage media of claim 12, wherein the instructions cause the one or more computing devices to perform operations further comprising: determining whether the current search query extends a current search conversation comprising the previous search query or starts a new search conversation with the user device.
 20. The one or more non-transitory storage media of claim 19, wherein the instructions cause the one or more computing devices to perform operations further comprising: in response to determining that the current search query starts a new conversation, generating a database query based on the current list of facet-value pairs. 